install: Enable installing to multi device parents by ckyrouac · Pull Request #1911 · bootc-dev/bootc

ckyrouac · 2026-01-14T19:54:35Z

When the root filesystem spans multiple backing devices (e.g., LVM across multiple disks), discover all parent devices and find ESP partitions on each. For bootupd/GRUB, install the bootloader to all devices with an ESP partition, enabling boot from any disk in a multi-disk setup. systemd-boot and zipl only support single-device configurations.

This adds a new integration test validating both single-ESP and dual-ESP multi-device scenarios.

Fixes: #481

Assisted-by: Claude Code (Opus 4.5)

gemini-code-assist

Code Review

This pull request successfully enables installing to multi-device parent filesystems, such as LVM spanning multiple disks. It correctly discovers all parent devices and, for bootupd/GRUB, installs the bootloader to all devices with an ESP partition. For bootloaders that only support single-device configurations like systemd-boot and zipl, the implementation correctly defaults to using the first available device. The changes are well-architected, adapting data structures and logic to handle multiple devices. A new, thorough integration test validates both single and dual ESP scenarios. Overall, this is a solid enhancement with good error handling and logging. I have one suggestion to further improve the robustness of ESP detection.

crates/lib/src/install.rs

crates/lib/src/bootloader.rs

crates/lib/src/bootc_composefs/boot.rs

ckyrouac · 2026-01-15T20:26:46Z

waiting to merge until the patch release goes out

crates/lib/src/bootc_composefs/boot.rs

HuijingHei · 2026-01-21T08:39:46Z

Maybe this PR could help to consider #1916 ? @ckyrouac

crates/lib/src/bootc_composefs/boot.rs

crates/lib/src/bootloader.rs

crates/lib/src/install.rs

HuijingHei · 2026-03-17T07:59:06Z

tmt/tests/booted/test-multi-device-esp.nu

+    # See https://tmt.readthedocs.io/en/stable/stories/features.html#reboot-during-test
+    match $env.TMT_REBOOT_COUNT? {
+        null | "0" => test_single_esp,
+        "1" => { test_dual_esp; test_three_devices_partial_esp; tmt-reboot },


How about add another test scenario like RAID1 that using the whole disks (see coreos/bootupd#1059), for example:

root@localhost-live:/home/fedora# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS vda 253:0 0 30G 0 disk └─md126 9:126 0 30G 0 raid1 ├─md126p1 259:0 0 477M 0 part ├─md126p2 259:1 0 954M 0 part /boot └─md126p3 259:2 0 28.6G 0 part / vdb 253:16 0 30G 0 disk └─md126 9:126 0 30G 0 raid1 ├─md126p1 259:0 0 477M 0 part ├─md126p2 259:1 0 954M 0 part /boot └─md126p3 259:2 0 28.6G 0 part /

To create RAID1 using command:
sudo mdadm -CR /dev/md126 -e 1 -l1 -n 2 /dev/loop0 /dev/loop1 --assume-clean

cgwalters

Looks sane to me, I have just nits really that are nonblocking, we can fix in followups too.

cgwalters · 2026-03-25T11:51:20Z

crates/lib/src/bootloader.rs

+        BwrapCmd::new(&target_root)
+            .setenv(
+                "PATH",
+                "/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/bin:/usr/local/sbin",


Let's use a shared const for this or perhaps better have set_default_path() on BwrapCmd

cgwalters · 2026-03-25T11:58:54Z

hack/provision-derived.sh

 CLOUDEOF
 fi

+# Temporary: update bootupd from @CoreOS/continuous copr until


This is a bit tricky as we probably also want to CI test the case without the new bootupd too...

I know, another matrix entry in theory...

I guess it'll be tested in post submits by the workflow test.

cgwalters · 2026-03-25T12:03:15Z

tmt/tests/booted/test-multi-device-esp.nu

+# Test that bootc install to-existing-root can find and use ESP partitions
+# when the root filesystem spans multiple backing devices (e.g., LVM across disks).
+#
+# Five scenarios are tested across three reboot cycles:


We're just testing installation to loopback, AFAICS there's no reason to reboot the host.

That said what would be a stronger test here is for us to test these setups probably via Anaconda where we boot into a guest VM with that. I tried to streamline this in bootc-dev/bcvk#202 but yeah it needs a bit of work.

I guess one thing we could do is extend the anaconda stuff we already have here in our integration tests.

Yet a different alternative is we ask tmt to directly support kickstart (and copy over some of the bcvk smarts there re mounting host container storage via virtiofs etc).

crates/lib/src/bootc_composefs/boot.rs

cgwalters · 2026-04-01T17:45:09Z

crates/lib/src/bootloader.rs

+/// files before reinstallation.  On multi-device setups only the first
+/// ESP is mounted and cleaned; stale files on additional ESPs are left
+/// in place (bootupd will overwrite them during installation).
+// TODO: clean all ESPs on multi-device setups


Hmm yeah maybe should move this logic into bootupd

cgwalters · 2026-04-01T17:46:06Z

hack/provision-derived.sh

+    *) copr_distro="centos-stream" ;;
+esac
+# Update bootc from rhcontainerbot copr; the new bootupd
+# requires a newer bootc than what ships in some base images.


But wait we're building this as part of our CI here right?

cgwalters · 2026-04-01T17:46:20Z

hack/provision-derived.sh

+enabled=1
+enabled_metadata=1
+REPOEOF
+dnf -y install bootupd-0.2.32.41.gb788553


Isn't this the main thing we need?

find_vmlinuz_initrd_duplicates() previously opened the absolute path /sysroot/state/deploy via STATE_DIR_ABS, which during install belongs to the host, not the target. Fix by passing a Dir opened relative to the target's physical_root. During fresh install the state dir naturally does not exist yet, so open_dir_optional returns None and the check is skipped -- no special-case guard needed. Also plumb physical_root through the setup_composefs_bls_boot match arms, and replace the hardcoded "/sysroot" in the Upgrade arm with storage.physical_root_path. Assisted-by: OpenCode (Claude Opus 4) Signed-off-by: Colin Walters <walters@verbum.org>

Assisted-by: Claude Code (Opus 4) Signed-off-by: ckyrouac <ckyrouac@redhat.com> Signed-off-by: Colin Walters <walters@verbum.org>

The composefs BLS and UKI boot setup paths called find_partition_of_esp() directly on the device, which fails when the root filesystem is on an LVM logical volume (the ESP is on the parent disk, not the LV). The store module had the same issue via require_single_root() + find_partition_of_esp(). Switch all call sites to find_colocated_esps() which walks up to the physical disk(s) via find_all_roots() before searching for the ESP, consistent with what install_systemd_boot and mount_esp_part already do. Assisted-by: Claude Code (Opus 4) Signed-off-by: ckyrouac <ckyrouac@redhat.com> Signed-off-by: Colin Walters <walters@verbum.org>

The test was using `get_target_image` which returns the upstream `docker://quay.io/centos-bootc/centos-bootc:stream10`. On composefs+grub variants provisioned with an updated bootupd from copr, the upstream image has stock bootupd with incompatible EFI update metadata, causing the install to fail with "Failed to find EFI update metadata". Switch to using `containers-storage:localhost/bootc` (the locally-built image), matching the pattern used by test-32, test-37, and test-38. The locally-built image has the updated bootupd with compatible metadata. Assisted-by: Claude Code (Opus 4) Signed-off-by: ckyrouac <ckyrouac@redhat.com> Signed-off-by: Colin Walters <walters@verbum.org>

The initial change to use locally-built images had two additional issues on composefs: 1. containers-storage: transport fails on composefs's read-only root with "mkdir /.local: read-only file system". Fix by exporting the image to an OCI layout directory on writable /var/tmp instead. 2. run_install() was masking /sysroot/ostree and removing bootupd update metadata, which composefs needs for bootloader installation and boot binaries. Fix by making run_install() skip these ostree-specific workarounds on composefs systems. Note: the composefs install-outside-container code path still has a separate bug ("Shared boot binaries not found" in boot.rs:745) that needs fixing in the Rust code. Assisted-by: Claude Code (Opus 4) Signed-off-by: ckyrouac <ckyrouac@redhat.com> Signed-off-by: Colin Walters <walters@verbum.org>

The multi-device ESP test creates ESP partitions and expects bootupd to install a UEFI bootloader. On BIOS-booted systems, bootupd instead tries to install GRUB for i386-pc, which requires a BIOS Boot Partition and fails. The test plan already requests UEFI provisioning via the hardware hint, but Testing Farm does not always honor this on CentOS Stream x86_64. Add a runtime check for /sys/firmware/efi so the test skips gracefully on BIOS hosts rather than failing. Assisted-by: Claude Code (Opus 4.6) Signed-off-by: ckyrouac <ckyrouac@redhat.com> Signed-off-by: Colin Walters <walters@verbum.org>

Extract the repeated PATH environment variable string into a set_default_path() method on BwrapCmd. The bwrap environment may not have a complete PATH, causing tools like bootupctl or sfdisk to not be found. This consolidates the workaround into one place. Assisted-by: OpenCode (Claude Opus 4) Signed-off-by: Colin Walters <walters@verbum.org>

Several improvements to ESP partition discovery: Add find_partition_of_esp_optional() returning Result<Option<&Device>> to cleanly separate three outcomes: found, absent, and genuinely unexpected errors (like unsupported partition table types). The existing find_partition_of_esp() is now a thin wrapper that converts None to Err. Add find_first_colocated_esp() helper to replace a 10-line pattern that was repeated verbatim 5 times across boot.rs and store/mod.rs. Deduplicate roots in find_all_roots() using a seen-set: in complex topologies like multipath, multiple parent branches can converge on the same physical disk. find_colocated_esps() now uses the optional variant to properly propagate real errors while treating absence normally. Also extract the match-on-if-else in setup_composefs_bls_boot into a let binding for readability. Assisted-by: OpenCode (Claude Opus 4) Signed-off-by: Colin Walters <walters@verbum.org>

The no-ESP test only checked for a non-zero exit code, which would also pass if podman itself failed for unrelated reasons. Check that the output contains "ESP" to confirm the right failure mode. Assisted-by: OpenCode (Claude Opus 4) Signed-off-by: Colin Walters <walters@verbum.org>

cgwalters · 2026-04-01T22:25:13Z

OK I pushed a few cleanups here, when you get back tomorrow can you look @ckyrouac ?

github-actions bot added the area/install Issues related to `bootc install` label Jan 14, 2026

bootc-bot bot requested a review from cgwalters January 14, 2026 19:54

gemini-code-assist bot reviewed Jan 14, 2026

View reviewed changes

crates/lib/src/install.rs Outdated Show resolved Hide resolved

cgwalters previously approved these changes Jan 15, 2026

View reviewed changes

crates/lib/src/bootloader.rs Outdated Show resolved Hide resolved

crates/lib/src/bootc_composefs/boot.rs Outdated Show resolved Hide resolved

crates/lib/src/bootc_composefs/boot.rs Outdated Show resolved Hide resolved

ckyrouac dismissed cgwalters’s stale review via f7b1892 January 15, 2026 20:26

ckyrouac force-pushed the multiple-backing-devices branch from f2a175a to f7b1892 Compare January 15, 2026 20:26

ckyrouac marked this pull request as draft January 15, 2026 20:26

ckyrouac force-pushed the multiple-backing-devices branch 3 times, most recently from 77b65cb to d03c6fa Compare January 16, 2026 17:31

cgwalters reviewed Jan 20, 2026

View reviewed changes

crates/lib/src/bootc_composefs/boot.rs Outdated Show resolved Hide resolved

cgwalters force-pushed the multiple-backing-devices branch from d03c6fa to 9b1c313 Compare January 21, 2026 13:08

ckyrouac force-pushed the multiple-backing-devices branch 2 times, most recently from 6802697 to 081f3b2 Compare January 22, 2026 15:29

ckyrouac marked this pull request as ready for review January 22, 2026 19:28

bootc-bot bot requested a review from jeckersb January 22, 2026 19:28

ckyrouac force-pushed the multiple-backing-devices branch from 081f3b2 to 9d0e284 Compare January 23, 2026 14:53

cgwalters requested changes Jan 26, 2026

View reviewed changes

crates/lib/src/bootc_composefs/boot.rs Outdated Show resolved Hide resolved

crates/lib/src/bootloader.rs Outdated Show resolved Hide resolved

crates/lib/src/install.rs Outdated Show resolved Hide resolved

cgwalters mentioned this pull request Jan 26, 2026

lib/install: Use mounted ESP for install to-filesystem #1953

Open

This was referenced Feb 11, 2026

Blockdev cleanup #2000

Merged

Blockdev bugfixes #2005

Merged

cgwalters mentioned this pull request Feb 12, 2026

Failure on RAID1 bootloader update coreos/bootupd#1059

Open

ckyrouac marked this pull request as draft February 16, 2026 17:12

ckyrouac force-pushed the multiple-backing-devices branch from 9d0e284 to 4ccc192 Compare March 11, 2026 19:40

HuijingHei reviewed Mar 17, 2026

View reviewed changes

ckyrouac force-pushed the multiple-backing-devices branch 2 times, most recently from 541691d to baa6c67 Compare March 24, 2026 19:17

ckyrouac requested a review from cgwalters March 24, 2026 19:17

ckyrouac marked this pull request as ready for review March 24, 2026 19:17

bootc-bot bot requested a review from jmarrero March 24, 2026 19:18

ckyrouac force-pushed the multiple-backing-devices branch 2 times, most recently from 8f69693 to 1794593 Compare March 25, 2026 11:36

cgwalters previously approved these changes Mar 25, 2026

View reviewed changes

ckyrouac dismissed cgwalters’s stale review via 0cb07e5 March 25, 2026 13:26

ckyrouac force-pushed the multiple-backing-devices branch 4 times, most recently from 9da140b to b7d4d61 Compare March 26, 2026 13:37

cgwalters previously approved these changes Apr 1, 2026

View reviewed changes

cgwalters mentioned this pull request Apr 1, 2026

composefs: Open deployments dir from physical root, not ambient path #2115

Open

cgwalters dismissed their stale review via b245dc6 April 1, 2026 22:23

cgwalters force-pushed the multiple-backing-devices branch from b7d4d61 to b245dc6 Compare April 1, 2026 22:23

cgwalters and others added 9 commits April 1, 2026 22:24

install: Enable installing to devices with multiple parents

fc6e9b7

Assisted-by: Claude Code (Opus 4) Signed-off-by: ckyrouac <ckyrouac@redhat.com> Signed-off-by: Colin Walters <walters@verbum.org>

cgwalters force-pushed the multiple-backing-devices branch from b245dc6 to 52602b4 Compare April 1, 2026 22:24

Conversation

ckyrouac commented Jan 14, 2026

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ckyrouac commented Jan 15, 2026

Uh oh!

Uh oh!

HuijingHei commented Jan 21, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

HuijingHei Mar 17, 2026

Choose a reason for hiding this comment

Uh oh!

cgwalters left a comment

Choose a reason for hiding this comment

Uh oh!

cgwalters Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

cgwalters Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

cgwalters Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cgwalters Apr 1, 2026

Choose a reason for hiding this comment

Uh oh!

cgwalters Apr 1, 2026

Choose a reason for hiding this comment

Uh oh!

cgwalters Apr 1, 2026

Choose a reason for hiding this comment

Uh oh!

cgwalters commented Apr 1, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants